Conflict of interest in the peer review process: A survey of peer review reports

Objectives To assess the extent to which peer reviewers and journals editors address study funding and authors’ conflicts of interests (COI). Also, we aimed to assess the extent to which peer reviewers and journals editors reported and commented on their own or each other’s COI. Study design and methods We conducted a systematic survey of original studies published in open access peer reviewed journals that publish their peer review reports. Using REDCap, we collected data in duplicate and independently from journals’ websites and articles’ peer review reports. Results We included a sample of original studies (N = 144) and a second one of randomized clinical trials (N = 115) RCTs. In both samples, and for the majority of studies, reviewers reported absence of COI (70% and 66%), while substantive percentages of reviewers did not report on COI (28% and 30%) and only small percentages reported any COI (2% and 4%). For both samples, none of the editors whose names were publicly posted reported on COI. The percentages of peer reviewers commenting on the study funding, authors’ COI, editors’ COI, or their own COI ranged between 0 and 2% in either one of the two samples. 25% and 7% of editors respectively in the two samples commented on study funding, while none commented on authors’ COI, peer reviewers’ COI, or their own COI. The percentages of authors commenting in their response letters on the study funding, peer reviewers’ COI, editors’ COI, or their own COI ranged between 0 and 3% in either one of the two samples. Conclusion The percentages of peer reviewers and journals editors who addressed study funding and authors’ COI and were extremely low. In addition, peer reviewers and journal editors rarely reported their own COI, or commented on their own or on each other’s COI.


Objectives
To assess the extent to which peer reviewers and journals editors address study funding and authors' conflicts of interests (COI). Also, we aimed to assess the extent to which peer reviewers and journals editors reported and commented on their own or each other's COI.

Study design and methods
We conducted a systematic survey of original studies published in open access peer reviewed journals that publish their peer review reports. Using REDCap, we collected data in duplicate and independently from journals' websites and articles' peer review reports.

Results
We included a sample of original studies (N=144) and a second one of randomized clinical trials (N=115) RCTs.
In both samples, and for the majority of studies, reviewers reported absence of COI (70% and 66%), while substantive percentages of reviewers did not report on COI (28% and 30%) and only small percentages reported any COI (2% and 4%). For both samples, none of the editors whose names were publicly posted reported on COI. The percentages of peer reviewers commenting on the study funding, authors' COI, editors' COI, or their own COI ranged between 0 and 2% in either one of the two samples. 25% and 7% of editors respectively in the two samples commented on study funding, while none commented on authors' COI, peer reviewers' COI, or their own COI. The percentages of authors commenting in their response letters on the study funding, peer reviewers' COI, editors' COI, or their own COI ranged between 0 and 3% in either one of the two samples.

Conclusion
The percentages of peer reviewers and journals editors who addressed study funding and authors' COI and were extremely low. In addition, peer reviewers and journal editors rarely reported their own COI, or commented on their own or on each other's COI.

Abstract:
Objectives: To assess the extent to which peer reviewers and journals editors address study funding and authors' conflicts of interests (COI). Also, we aimed to assess the extent to which peer reviewers and journals editors reported and commented on their own or each other's COI.

Study design and methods: We conducted a systematic survey of original studies published
in open access peer reviewed journals that publish their peer review reports. Using REDCap, we collected data in duplicate and independently from journals' websites and articles' peer review reports.

Results:
We included a sample of original studies (N=144) and a second one of randomized clinical trials (N=115) RCTs.
In both samples, and for the majority of studies, reviewers reported absence of COI (70% and 66%), while substantive percentages of reviewers did not report on COI (28% and 30%) and only small percentages reported any COI (2% and 4%). For both samples, none of the editors whose names were publicly posted reported on COI. The percentages of peer reviewers commenting on the study funding, authors' COI, editors' COI, or their own COI ranged between 0 and 2% in either one of the two samples. 25% and 7% of editors respectively in the two samples commented on study funding, while none commented on authors' COI, peer reviewers' COI, or their own COI. The percentages of authors commenting in their response letters on the study funding, peer reviewers' COI, editors' COI, or their own COI ranged between 0 and 3% in either one of the two samples.

Conclusion:
The percentages of peer reviewers and journals editors who addressed study funding and authors' COI and were extremely low. In addition, peer reviewers and journal editors rarely reported their own COI, or commented on their own or on each other's COI.
Keywords: Conflict of Interest, funding, authors, peer reviewers, editors, peer review process, editorial process

Word count: 2613
Ethical approval: The study involves no human subjects and requires no ethical approval.
What is new?  This is one of the first studies that examines the extent to which peer reviewers and editors consider study funding and authors' COI, and the extent to which they report on their own COI.  The percentages of peer reviewers and journals editors who addressed study funding & authors' COI were extremely low.  Peer reviewers and journal editors rarely reported their own COI, or commented on their own or on each other's COI.

Introduction
The Institute of Medicine (IOM) defined conflict of interest (COI) as "circumstances that create a risk that professional judgments or actions regarding a primary interest will be unduly influenced by a secondary interest" [1, 2]. In the research field, conflict of interest arises when an individual prefers, or is perceived to prefer, their own interests which would increase the risk of biasing their judgments related to their professional obligations and responsibilities.
COI may involve a broad spectrum of interests. The financial interests are the most obvious.
For example, a researcher may receive significant financial rewards from a pharmaceutical company with interest in the findings of their research. Such financial COIs are common among members of clinical guidelines panels [3,4]. There is evidence that the quality of the research as well as guidelines may be negatively affected [5].
Non-financial interests can also affect the integrity of research. Such interests include career advancement, fame, social interests, and intellectual beliefs [6]. For instance, an editor may be conflicted when peer reviewing a colleague's work. Intellectual COI is another type of nonfinancial COI discussed as far as two decades ago [7]. Lately, intellectual COI has been increasingly acknowledged [8] particularly in clinical practice guidelines (CPG) development [9,10]. It has been defined as ''academic activities that create the potential for an attachment to a specific point of view that could unduly affect an individual's judgment about a specific recommendation'' [11].
Researchers are expected to avoid and minimize COIs, and disclose them when they exist. The International Committee of Medical Journal Editors (ICMJE) has developed a specific and unified form for disclosure of COIs to facilitate and standardize authors' disclosures.
Moreover, there are recommendations on how to manage and declare COIs among authors of clinical practice [12].
Peer reviewers and editors of journals play a key role in assessing and publishing research manuscripts [13]. On one hand, they need to assess the COI of authors and the funding source of the study. On the other hand, journal peer reviewers and editors may have their own conflict of interests that need to be disclosed [14].
In the current era, the standard for scientific publishing is to have research findings evaluated and published through a peer review process [15]. Peer-reviewed biomedical journals are publishing enormous number of articles each year. As of 2012, about 28000 scholarly journals published more than 2 million peer-reviewed articles [16]. An optimal scientific peer review process is essential to maintain the integrity of the scientific research and to support evidence-based practice.
There is increased media attention to the reported conflicts and concerns about the impact of industry-sponsorship [17,18]. A transparent handling of conflicts of interest is essential for the public trust in the scientific process and the credibility of peer-reviewed published articles [6]. The reporting of authors' disclosure of conflicts of interest in publications has become the standard [14]. In order to facilitate and standardize the process of authors' disclosures, the International Council of Medical Journal Editors (ICMJE) has developed a form for the disclosure of COI [6]. The authors are required to declare via this form all financial and nonfinancial benefits or personal relationships that might bias their work.
COI issues are relevant to all participants in the peer-reviewed publication process -including peer reviewers, editors, and the editorial board members of the journals. The peer reviewers' role is to critically assess the manuscript, by constructively commenting on the scientific work, and suggesting how to improve it [19]. Moreover, peer-reviewers and editors are expected to reflect and comment on the authors' disclosures of conflicts of interest. There are questions about the effectiveness of the current system of COI disclosures. A randomized controlled study found that providing journal reviewers with authors' conflict of interest information had no significant effect on their rating of the quality of the manuscript [20].
Similarly, journal editors have a core role in managing the review process, through assessing the peer reviewers' reports and making the final decision on acceptance for publication. In addition, editors can significantly impact the integrity, quality and fairness of the peer review process by how they select the peer reviewers and managing any misconducts by authors or reviewers [21]. Editors are also expected to review and consider the authors' disclosures of conflict of interest as part of the peer review process. However, this aspect of their role has not been studied yet.
According to the ICMJE, peer reviewers and editors have to disclose their own conflicts of interest [6]. In some cases, those invited to peer review and editors may need to rescue themselves from being involved. Little is known about the practices and policies of journals regarding disclosures of conflict of interest among peer-reviewers and editors to public [14].
A study assessing the COI policies of health policy and services journals, found only one that described how the COIs of the editorial team are managed during the editorial process [22].
The objective of this study was to assess the extent to which peer reviewers and journal editors address authors' conflicts of interests and study funding. Also, we aimed to assess the extent to which peer reviewers and journals editors reported and commented on their own or each other's COI.

Methods:
We included two samples in this study. First, we included a sample with any type of original research. However, we found that a very low percentage (9%) had at least one author reporting presence of COI. Given this would not allow us achieve the study objective, we decided to collect a second sample restricted to randomized controlled trials (RCTs) as a survey had found that more than half of clinical trials had authors reporting presence of COI [23].

Eligibility criteria
We included journals that publish in the health field and in English, are indexed in Medline, and publish all their peer review reports. We included only original articles that are peer reviewed.

Selecting reports
We used the Directory of Open Access Journals (DOAJ) as an initial list of journals. Then, we filtered the journals by language (English) and subject (Medicine). Then, we selected our samples according to the eligibility criteria. For our first sample, we included the latest two original publications from each journal that had peer review reports. For our second sample, we included the latest two randomized controlled trials (RCTs) that had peer review reports.
Teams of two reviewers implemented the selection process in duplicate and independently.
They resolved any disagreements through discussion, or with the help of a third reviewer if needed.

Data abstraction
We developed data extraction form using the Research Electronic Data Capture (REDCap) tool hosted at the American University of Beirut. We also developed detailed instructions. After conducting calibration exercises, teams of two reviewers abstracted data in duplicate and independently. They resolved any disagreements through discussion, or with the help of a third reviewer if needed.
We abstracted the following data:

Data analysis
We exported all data from REDCap to an Excel sheet for data cleaning and consistency checks.
We conducted quantitative descriptive analyses of all variables. We used percentage for categorical variables and median and interquartile range for continuous variables. We also conducted a thematic analysis of comments extracted.   Table 1 presents the general characteristics of the journals included in the two samples. For both samples, the majority of journal were from the clinical field (80% and 83%), were published in BioMed Central (BMC) (85% and 89%), and were indexed in Medline (97% and 100%). For the majority of journals peer review was by invitation (99% and 98%), was conducted pre-publication (90% and 91%), and did not reveal the identity of the peer reviewers (56% and 64%). Table 2 presents the general characteristics of publications included in the two samples. The percentages of RCTs were 3% for sample 1 and 100% for sample 2. For both samples small percentages had COVID-19 as topic (8% and 3%), the median number of authors was 6, the majority had their first author primarily affiliated with academia (87% and 93%), and the majority had the first author affiliated with a high-income country (63% and 61%). Table 3 presents characteristics of the peer review process. For both samples, the median for number of rounds of peer review, and the number of rounds of revision, and the number of peer reviewers was 2. In the two samples, the median of the number of editors involved was 1. Only 11% and 13% of editors' letters to the authors were posted publicly, and of those 63% and 53% respectively provided the name of the editor. Table 4 presents the characteristics of funding of the included studies. For both samples, the majority of studies were reported as funded (67%and 90%) and included a statement on the funder's role (57% and 59%). The top two sources of funding were governmental (61% and 46%) and internal funding (41% and 46%). Table 5 summarizes the declaration of conflict of interest by authors, peer reviewers, and editors. For both samples, the majority of studies had all authors report the absence of COI (89% and 68%). For both samples, the majority of studies had reviewers identified by name (55% and 56%), reported absence of COI (70% and 66%). Substantive percentages did not report on COI of reviewers (28% and 30%) while only small percentages reported any COI (2% and 4%). For both samples, none of the editors whose names were publicly posted reported on COI. Table 6 summarizes the extent of authors, peer reviewers and editors commenting on study funding and conflicts of Interest. The percentages of peer reviewers commenting on the study funding, authors' COI, editors' COI, or their own COI ranged between 0 and 2% in either one of the two samples. 25% and 7% of editors respectively in the two samples commented on study funding, while none commented on authors' COI, peer reviewers' COI, or their own COI.

Commenting on declarations of conflict of interest
The percentages of authors commenting in their response letters on the study funding, peer reviewers' COI, editors' COI, or their own COI ranged between 0 and 3% in either one of the two samples. Figure 2 below illustrates these results among our 2 samples. Figure 2: Commenting on own and other's COI by authors, peer reviewers, and editors 4. Discussion:

4.1.Summary of findings
In summary, the percentages of peer reviewers and editors commenting on study funding and authors' COI were extremely low. In addition, peer reviewers and journal editors rarely reported their own COI, or commented on their own COI, or on each other's COI.

4.2.Strengths and limitations
To our knowledge, this is one of the first studies that examines the extent to which peer reviewers and editors consider study funding and authors' COI, and the extent to which they report on their own COI. To ensure our findings are informative, we went beyond our original plan and included a second sample that involved a higher percentage of articles whose authors reported presence of COI, compared with our first sample. We found consistent results for the two samples. One limitation of this study is that our samples represent the peer review processes of articles that ended up being accepted for publication, but not of those that were not published or those that were published in non-open peer review journal.

4.3.Comparison to similar studies
The percentage of articles with at least one author reporting presence of COI for our first sample was only 9%. This percentage is much lower compared to that of systematic reviews (41%) and clinical trials (57%) [23,24]. For our second sample, the percentage was higher at 30% because we included only randomized clinical trials. In addition, our findings that peer reviewers and editors are not reporting their own COI to a good extent are consistent with other studies in the literature. A recent study highlighted the lack of discussion surrounding editorial conflicts of interest in public health journals [25]. The authors noted that the editorial COI policies did not adhere to guidelines and that these policies need improvements [25].

4.4.Implications for practice and research
The results of our project provide insight about the COI disclosure of peer reviewers, authors and editors. This may help journals to develop policies to improve how COI is declared and managed during their editorial processes. It would be relevant to conduct qualitative research to explore why some peer reviewers and editors are commenting on authors' COI and on the study funding and others are not. Such study may help in developing strategies to improve COI declaration and management during the peer review process.